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Surveying Assessment in Experiential Learning: A Single Campus Study 


Abstract 

The purpose of this study was to determine the methods of experiential assessment in use at a Canadian 
university and the extent to which they are used. Exploring experiential assessment will allow identification of 
commonly used methods and facilitate the development of best practices of assessment in the context of 
experiential learning (EL) at an institutional level. The origins of EL are found in the work of Dewey (1938), 
later modified by Kolb and Fry (1975). Experiential methods include: experiential education, service learning 
problem-based learning and others such as action learning, enquiry-based learning, and case studies. Faculty 
currently involved in EL at the participating university were invited to complete an online survey about their 
teaching and assessment methods. This paper will share the results and analysis of the EL inventory survey. 

Lobjectif de cette etude etait de determiner quelles methodes d evaluation experientielle sont employees dans 
les universites canadiennes et dans quelle mesure elles sont employees. Le fait d explorer revaluation 
experientielle permettra d’identifier quelles sont les methodes employees couramment et facilitera le 
developpement des meilleures pratiques d evaluation dans le contexte de I’apprentissage experientiel au niveau 
institutionnel. Les origines de I’apprentissage experientiel se trouvent dans les travaux de Dewey (1938), 
modifies plus tardpar Kolb et Fry (1975). Les methodes experientielles comprennent: leducation 
experientielle, I’apprentissage par le service, I’apprentissage par problemes, ainsi que quelques autres tel que 
I’apprentissage par action, I’apprentissage par I’enquete et les etudes de cas. Les professeurs qui pratiquent 
actuellement I’apprentissage experientiel dans les universites participantes ont ete invites a remplir un 
questionnaire en ligne portant sur leur enseignement et leurs methodes d evaluation. Get article partage les 
resultats et les analyses du sondage sur I’apprentissage experientiel. 
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Universities across Canada are looking for ways to make learning more meaningful and 
impactful for students to increase student engagement in their education and better prepare them 
for the workforce. One of the areas identified to address this important goal is Experiential 
Learning (EL). Preparing students for the workforce by engaging them in more meaningful 
learning experiences is also the goal of our institution, a Canadian university. To this end, the 
purpose of this study was to determine the forms of EL assessment in use at our institution, and 
to gauge the extent to which these methods have infiltrated teaching. It is our belief that 
informing our practice will lead to change. Identification of commonly used methods will 
facilitate the development of institutional efforts to scale up experiential assessment in keeping 
with EL. Also, this knowledge will provide a base upon which to study the impact of the choice 
of assessment method (experiential or traditional) on student learning in courses that use EL 
methods. This study establishes a baseline for the range of assessment methods used by 
instructors teaching courses with a component of EL. It also identifies (and normalizes) 
alternative assessment practices in a single campus environment. 

EL is a term that refers to learning by doing (Kolb & Ery, 1975), and is further defined as 
authentic, student centered, hands-on, and situated in relevant learning contexts (Kolb & Kolb, 
2005). Due to its effectiveness for learner success, EL is becoming an option for those teaching 
at the post-secondary level (Breunig, 2005; King, 1993; Nnodim, 1997). The main driver behind 
these approaches to learning is the need to have students working outside of “traditional” 
classroom settings and procedures. 

EL may include a variety of different instructional methodologies such as field work, 
internship, presentation, experiential education, and problem-based learning, to name a few. 
Although there is anecdotal and research evidence that illustrates many of these methodologies 
are being applied, EL is still not widespread. Even more importantly, the assessment and 
evaluation tied to EL is less well-defined. To better understand EL methodologies and 
assessment at our institution, an inventory of existing options needed to be determined. In an 
attempt to be comprehensive and inform our investigation of these methods, the review of the 
literature began examining assessment methods in general. This initial review was then refined to 
create an inventory of both the application and assessment of EL with which to inform a survey 
of faculty at a Canadian university to determine the EL methodologies that were being used and 
how the students were being assessed. This paper shares the process and the outcomes of the 
survey. 

Review of the Relevant Literature 


Experiential Learning 

The origins of EL are found in the work of Dewey (1938) who identified the role of 
experience and reflection in learning. This initial description of EL was refined by Kolb and Ery 
(1975) into a cyclic model of four stages: concrete experience, observation and reflection, 
concept formation, and re-evaluation. Since the introduction of the Kolb and Ery model, a range 
of EL teaching concepts and methods have been developed to facilitate this form of learning. 
Examples include: experiential education, i.e., learning through direct experience (Bruenig, 
2005), students working with community partners or service learning (Grossman, Patel, & 
Drinkwater, 2010), working cooperatively within groups to find solutions to real problems or 
problem-based learning (Bethell & Morgan, 2011), and others such as action learning 
(discussion, role playing, debates, writing and reflecting), inquiry based learning, and case 
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studies (Quinn & Shurville, 2009). These methods include EL in forms as simple as a group 
discussion or using the outdoors as a classroom. EL employs strategies such as teacher modeling 
(Hasten, 2007), think-aloud (Neilsen, 2002), and reciprocal-peer teaching (Meister, 2012; 
Doolittle, Hicks, Triplett, Nichols, & Young, 2006; Krych, March, Bryan, Peake, Pawlina, & 
Carmichael, 2005). EL is foremost a philosophy based on the work of Dewey (1938) and, by 
subscribing to that theory, EL methods have been repeatedly recognized as superior to traditional 
teaching methods in the enhancement of student performance and the promotion of critical 
learning (Breunig, 2005; King, 1993; Nnodim, 1997). 

Literature on EL presents many studies that assess the impact of applying experiential 
theory and teaching methods on student understanding and retention. Kolb and Kolb (2005) view 
EL as a process where the transformation of experience results in the creation of new knowledge. 
Students take an active learning role (Dennehy, Sims, & Collins, 1998) and communication is 
two-way between instructor and student, and between students (Van Eynde & Spencer, 1988). 
By contrast, in a lecture-based course concepts and skills are described, thought involves one¬ 
way communication between instructor and student, and the students take a passive learning role. 
It is believed that, due to these differences, EL accommodates a range of learning styles (Rhodes 
& Roessner, 2009), whereas lecture based learning does not. EL has been shown to promote deep 
learning (Bethell & Morgan, 2011; Sternberg & Zhang, 2001). Deep learning involves relating 
new ideas to previous knowledge and relating concepts to everyday life. A surface approach is 
where information is transferred to students with the objective of learning with a focus on 
individual elements instead of integration (Sternberg & Zhang, 2001). EL can accommodate a 
range of learning styles (Breunig, 2005), whereas traditional teaching models promote primarily 
one means of education: surface learning (Anderson, 1998). 

Assessment 

While many studies have demonstrated the positive impact of EL methods on student 
learning outcomes, the choice of evaluation method used to assess students in courses that use 
EL are much less studied. A primary purpose of assessment is to validate the knowledge and 
skill level of a student for the benefit of the intended profession and society (Quinn & Shurville, 
2009). Assessment frames student learning and can, in some cases, have more impact on the 
learning process than teaching (Gibbs, 2006). Traditional testing has the potential to be a 
distorted measure of learning because of the nature of the test itself (Gibbs, 2006). Our review of 
the literature on assessment identified a range of commonly used methods that were either 
experiential or traditional as defined by the authors (see Table 1). By comparison, the range of 
traditional assessment methods commonly used is narrow, whereas the range of experiential 
assessment methods are broad. 
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Table 1 


Assessment Methods and their Classification as Traditional or Experiential According to the 
Literature 


Assessment method 

Type 

Reference(s) 

Paper/essay 

Traditional 

Struyven, Dochy, & Janssens (2008a); Struyven, 

Dochy, & Janssens (2005); van de Watering, Gijbels, 
Dochy, & van der Rijt, (2008) describe papers/projects 
as alternative assessment methods (contrasted with oral 
and written exams) 

Written exam 

Traditional 

Struyven, Dochy, & Janssens (2008a); Davis (1988); 
van de Watering, Gijbels, Dochy, & van der Rijt, 

(2008) 

Multiple-choice 

exam 

Traditional 

Struyven, Dochy, & Janssens (2008a); Struyven, 

Dochy, & Janssens (2005); Struyven, Dochy, Janssens, 

& Gielen (2008a); Struyven, Dochy, Janssens, & 

Gielen (2008b) 

Practical 

exam/performance 

test 

Experiential 

Struyven, Dochy, & Janssens (2008a); Davis (1988) 

Oral exam 

Traditional 

or 

experiential 

Struyven, Dochy, & Janssens (2008a) describe oral 
exams taken as a group as an unconventional or 
alternative assessment method, and consider individual 
oral closed book exams as a traditional assessment 
method; van de Watering (2008) contrast both 
individual and group oral exams with other alternative 
assessment methods 

Portfolio 

assessment 

Experiential 

Struyven, Dochy, & Janssens (2008a); Struyven, 

Dochy, & Janssens (2005); Qualters (2010); Struyven, 
Dochy, Janssens, & Gielen (2008a); Struyven, Dochy, 
Janssens, & Gielen (2008b); van de Watering, Gijbels, 
Dochy, & van der Rijt, (2008) 

Take-home exam 

Traditional 

or 

experiential 

Struyven, Dochy, & Janssens (2008a); van de 

Watering, Gijbels, Dochy, & van der Rijt, (2008) 
contrast written take home exams with other alternative 
assessment methods 

Open-book exam 

Traditional 

or 

experiential 

Struyven, Dochy, & Janssens (2008a) classify 
individual written open-book exams as traditional; 
Williams & Wong (2009) describe open-book, open 
web (OBOW) exams as an alternative to closed book, 
invigilated alternatives, with OBOW exams being more 
aligned with constructivist learning theory 

Case-based exam 

Experiential 

Struyven, Dochy, & Janssens (2008a); Struyven, 

Dochy, & Janssens (2005); Struyven, Dochy, Janssens, 

& Gielen (2008a); Struyven, Dochy, Janssens, & 

Gielen (2008b) 
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Table 1 (continued) 


Assessment method 

Type 

Reference(s) 

Journal assessment 

Experiential 

Breunig (2005); O’Toole (2007) 

Presentation 

Experiential 

Breunig (2005); Struyven, Dochy, & Janssens (2005 

Simulation/role¬ 

Experiential 

~ Struyven, Dochy, & Janssens (2008a); Struyven, 

playing 


Dochy, & Janssens (2005); Davis (1988) 

Debriefing interview 

Experiential 

Davis (1988) 

Product assessment 

Experiential 

Davis (1988) 

Concept mapping 

Experiential 

Ruiz-Primo, Shavelson, Ei, & Schultz (2001) 

Project report 

Experiential 

van de Watering, Gijbels, Dochy, & van der Rijt, 

(2008) 

Narrative 

Experiential 


assessment 

Other 

Experiential 

Critical reflection paper (Breunig, 2005); Wikis and 
forums (Puente, 2007); Computerized tests (van de 
Watering, Gijbels, Dochy, & van der Rijt, 2008) 

Assessment Style 

Type 

Reference(s) 

Individual task 

Traditional 

Struyven, Dochy, & Janssens (2008a) 

Group task 

Experiential 

Struyven, Dochy, & Janssens (2008a) 

Evaluation Style 

Type 

Reference(s) 

Instructor 

Traditional 

Stefani (1992) 

assessment 

Self-assessment 

Experiential 

Struyven, Dochy, & Janssens (2008a); Breunig 
(2005); Struyven, Dochy, & Janssens (2005); Davis 
(1988); Stefani (1992) 

Peer assessment 

Experiential 

Struyven, Dochy, & Janssens (2008a); Breunig 
(2005); Struyven, Dochy, & Janssens (2005); Stefani 
(1992); Struyven, Dochy, Janssens, & Gielen (2008a); 
Struyven, Dochy, Janssens, & Gielen (2008b) 

Collaborative 

Experiential 

Struyven, Dochy, & Janssens (2008a); Stefani (1992); 

assessment 


O’Toole (2007) 


* Incudes assessment methods described as “experiential,” “alternative,” or “authentic,” or assessment methods 
contrasted with “traditional” methods. 


Anderson (1998) gives us further proof for the distinction between traditional and 
alternative assessment, comparing philosophical beliefs and theoretical assumptions of 
traditional versus alternative assessment (see Figure 1). The comparison illustrates the theoretical 
interpretation of nine underlying philosophical tenets of the assessment of learning. For example, 
traditional assessment assumes knowledge can only be interpreted as a single truth whereas 
alternative assessment allows for multiple interpretations of knowledge. Traditional assessment 
is passive rather than active. Power and control of learning is hierarchical in traditional 
assessment, but shared in alternative assessment. Although identified by Anderson as alternative, 
those methods would also fit the criteria for experiential assessment. 
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Traditional Assessment 


Alternative Assessment 

Universal Assessment 

Knowledge 

Multiple meanings 

Passive process 

Learning 

Active process 

Separate product from 

Process 

Emphasizes product and 

process 

process 

Discrete isolated bits of 
information 

Focus 

Focus on inquiry 

To document learning 

Purpose 

To facilitate learning 

Cognitive abilities as separate 
from affective and conative 
abilities 

Abilities 

Connects between cognitive, 
affective and conative abilities 

Views assessment as 
objective, value-free and 
neutral 

Assessment 

Views assessment as 
subjective and value-laden 

Hierarchical model 

Power and Control 

Shared model 

Learning as an individual 

Individual versus 

Learning as a collaborative 

process 

Collaborative processes 

process 


Figure 1. Comparison of philosophical beliefs and theoretical assumptions of traditional and alternative assessment 
(adapted from Anderson, 1998). 


Experiential Assessment 

Most instructors will teach and assess based on their own experience (Quinn & Shurville, 
2009) and, as a result, tend to use traditional testing methods in EL courses. For example, 
although several studies use a range of assessment methods in EL courses such as journals, 
reports, group projects, presentations, self-evaluation, etc., these studies also still rely on 
traditional assessment formats such as tests and quizzes (Breunig, 2005; Garvin & Ramsier, 
2003; Rhodes & Roessner, 2009). The persistence of the use of traditional assessment methods in 
EL courses may be due to instructor perception of traditional methods, such as written 
assessment, as the most effective methods for evaluating student learning (Davis 1988). 
However, in a comparison of perceptions regarding what was taught, practiced, and assessed in 
classroom EL course, Haigh & Kilmartin (1999) found that students perceived differences 
between what was being taught and practiced, and how they were being assessed. This suggests 
that alignment of assessment methods with teaching strategies in EL courses may be inhibited by 
a lack of understanding of EL methods. Currently, there is little discussion in the literature on the 
use of traditional type testing methods in EL courses and the impact on student learning 
outcomes. As well, there is little discussion on the appropriate types of assessment methods for 
EL courses and on matching the assessment method with the form of EL. 
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Breunig (2005), who used mixed methods approaches in experiential education to address 
multiple learning styles, also indicated that a multiple methods approach to assessment was also 
needed. She proposed evaluating student performance using methods such as journals and 
presentation in addition to traditional assessment methods. Stefani (1992) demonstrated that 
peer-assessment and self-assessment produced similar results when compared to traditional 
assessment methods. Puente (2007) evaluated student performance in group activities using 
rubrics to assess student contributions to wikis. He found several advantages to interacting with 
and providing feedback to students using wikis; however, Puente made note that assessment 
tended to focus on quantity rather that quality. Understanding why, what, and how to assess is an 
important step in developing an effective assessment tool according to Quakers (2010). He 
indicated that the key to assessment is the instructor deciding on the “burning question(s)” that 
need to be answered and knowing this will guide development of meaningful assessment 
methods. The author also recommended the use of mixed methods of assessment matched with 
EL. He presented a model for assessment that evaluated student skills before, during and after 
the learning experience (Input, Experience, Output model: TE-0). Student surveys were used to 
evaluate input and output, with reflective journals and observation used to assess students during 
the experience. It is important to develop and use methods of assessment that are compatible 
with the concept of deep learning and varied learning styles. McGuire & Banerjee (2008) see the 
flexibility and variety of EL as positive for learners and this flexibility can be transferred to 
evaluation. 

Student perception of assessment method can impact student performance. Struyven, 
Dochy, & Janssens (2005), in a study of assessment methods in student-activated learning 
courses, found that students expecting to be evaluated using essay-type assessment tools 
performed better on multiple choice exams than students expecting to be evaluated using the 
multiple choice exams. Their conclusion was that the perception of essay-type methods 
promoted deeper learning and the expectation of multiple choice methods promoted surface 
learning in students in spite of the fact that both groups had a preference for multiple choice 
exams. Ironically, students also felt that multiple choice exams were an inappropriate assessment 
method (Struyven, Dochy, & Janssens, 2005). Struyven, Dochy, Janssens, Schelfhou, & Gielen 
(2006) compared a set of EL assessment methods to a traditional, multiple choice evaluation of 
student performance. Student performance was highest when evaluated using multiple choice 
regardless of whether the teaching was lecture based or experiential. It is possible that the 
conditions of the examination - multiple choice being closed book compared to open book for 
the experiential methods - had an impact on the results (Struyven, Dochy, Janssens, Schelfhou, 
& Gielen, 2006). 

Interestingly, Struyven, Dochy, & Janssens (2008a) contradicted their previous work 
(Struyven, Dochy, & Janssens, 2005) by finding that student preferences were in-line with 
students’ perception of the appropriateness of the assessment method. Consistent with Struyven, 
Dochy, & Janssens (2008a), students perceived multiple choice exams as less appropriate than 
more experiential methods such as portfolio assignment or case-based assessment; however, 
students perceived student-activated learning to be lower quality instruction. It was felt that these 
perceptions were possibly due to a mismatch between student learning habits and the student- 
activated learning environment. This unfamiliarity may be the reason for the conclusions drawn 
by Struyven, Dochy, Janssens, & Gielen (2008a), which found that students’ learning was not 
deepened by the use of student-activated learning and assessment. The authors felt that the 
methods promoted surface learning instead of deep learning because students perceived that 
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learning in this environment required more work for less feedback and structure. That familiar, 
essay-type assessment promoted deeper learning in Struyven, Dochy, & Janssens (2005), and 
that unfamiliar experiential assessment methods such as portfolio assignment, as used in 
Struyven, Dochy, Janssens & Gielen (2008a) did not indicate that it is important to choose an 
experiential assessment method that students may have been exposed to before or possibly 
matched to the learning experience itself. Overall, it appears important that students prefer the 
EL experience as this leads to greater perceived learning and higher scores when assessed 
(Struyven, Dochy, & Janssens, 2008b). 

Although several studies use a range of assessment methods in EL courses such as 
journals, reports, group projects, presentations, self-evaluation, etc., these studies also still rely 
on traditional assessment formats such as tests and quizzes (Breunig, 2005; Garvin & Ramsier, 
2003; Rhodes & Roessner, 2009). Currently, there is little discussion in the literature on the use 
of traditional type testing methods in EL courses and the impact on student learning outcomes. 
As well, there is little discussion on the appropriate types of assessment methods for EL courses 
and on matching the assessment method with the form of EL. 

Quinn & Shurville (2009) called for academic institutions to “scale up assessment of 
experiential learning to the institutional level” (p. 330) in the same way that this was done for EL 
itself. They are in part responding to the idea of educating the public for a knowledge-based 
economy and the need for a pedagogy to transfer the required skills, knowledge, and creativity 
(Hartley, 2003). Thus, teaching for a knowledge-based economy requires a move away from 
teaching-centered (traditional) methods to methods that encourage self-reliance and creativity 
(Hartley, 2003) or methods that promote deep learning and use higher-level cognitive strategies, 
as do EL methods. In the same way that early adopters initiated institutional level EL in 
educational institutions, experiential assessment methods are at a stage where early adopters 
have introduced a variety of methods into the pedagogy of their discipline, and have positioned 
their institutions to “scale up” assessment to the institutional level. 

Based on our review of the literature, there is clear differentiation between traditional and 
experiential methods of assessment. Despite the need to match teaching and assessment methods, 
methods of assessment in EL courses often remain traditional. We found a paucity of 
information regarding the methods of assessment currently being used in EL courses. 

Research Context 


Our host institution set a target to expand EL for all students by 20% (University 
Planning Document, 2012), a commitment expressed in its latest integrated plan in response to 
its students expressing a desire for more EL activities and courses that address their diverse 
learning styles. The integrated plan for our host institution was a document articulating several 
institutional level themes including a commitment to Innovation in Academic Programs and 
Services. The target to expand EL by 20% was an institutional initiative under this theme. 
Notably, this target is not atypical of other universities across Canada. Selected examples include 
Ryerson University’s institutional commitment to EL complete with faculty supports and 
frameworks, University of Waterloo’s co-operative learning programs, which are a large part of 
their student experience, and Dalhousie University’s course offerings that are specifically about 
the development of understanding in EL for learners. Along with other Canadian institutions, our 
host institution has made a concerted effort to promote and expand its use of EL and offers a 
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number of programs that have EL opportunities; as sueh, it is an appropriate eontext to study EL 
as it relates to teaehing and learning in Canadian higher edueation. 

The university in this study offers a number of programs that have EL opportunities. 
Early adoption by faculty and effective communication and promotion of EL methods has 
resulted in a range of EL opportunities for students across campus. The participating university is 
well positioned to develop an institutional level program for delivery of EL. Less developed is 
the state of assessment in EL courses. It is assumed that early adopters have also employed a 
range of assessment methods; however, it is not known to what extent such methods are used in 
terms of the types used, the contribution to overall assessment, and the number of instructors that 
use these methods. As well, to what extent traditional assessment methods are used in courses 
that are experiential is also unknown. 

We believe this inventory will facilitate the development of institutional efforts to scale 
up experiential assessment in keeping with EL. Also, this knowledge will provide a base for 
future study of experiential assessment methods in courses that use EL methods. The specific 
objectives of this research were to conduct a campus-wide survey of assessment methods used in 
EL courses at our university and to inform the broader educational community of assessment 
method options for use in EL courses. 


Method 

An initial review of the literature identified the range of possible EL methods and 
possible EL assessment methods. To determine what was currently taking place on campus in 
terms of EL and assessment, the researchers developed and utilized an online survey instrument. 
Although many teaching and assessment methods were identified, no one publication presented a 
comprehensive instrument to conduct an inventory. However, the survey developed by Davis 
(1988) to study the development of assessment in experiential education [EL] provided examples 
of relevant survey questions and served as a guide to the development of the questions for our 
own survey. 

The specific methodologies identified in the literature review were used to guide 
development of the online survey using the online survey creation software Eluid Surveys v.4.0 
(Eluidsurveys, 2013). The survey was designed and piloted with university faculty and 
instructional designers fa mi liar with EL. After the expert review, minor changes were made to 
the instrument to make the questions clearer. The final version of the instrument (see the 
Appendix) consisted of 11 to 18 questions asking participants to self-assess the level of EL in 
their course. The number of questions in the set depended on where the respondent’s course was 
situated on a range from entirely experiential to entirely traditional in terms of teaching method. 
Participants, in addition to indicating their degree of experiential activity, also indicated specific 
teaching and assessment methodologies and, where applicable, ranked their choices. Survey 
questions were designed so that respondents were free to choose any or all responses that 
applied. The survey was part of a larger study that was approved on ethical grounds by the 
research ethics board at the researchers’ university. 

Staff at the participating university’s learning center who are dedicated the promotion of 
innovation in teaching provided a comprehensive list of instructors who had previously self- 
identified as using EL teaching and assessment methods. The list of 463 instructors was used to 
distribute the online survey. Participants were invited over a three-week period to complete the 
survey. 
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A variety of analyses were conducted after the data was collected. Descriptive statistics 
were used to identify significant trends in teaching methods and assessment. Additionally, tests 
for multiple marginal independence (MMI) were used to examine relationships between course 
types - a single-response categorical variable (SRCV) - and assessment methods used - a 
multiple-response categorical variable (MRCV). Tests of MMI are equivalent to the Pearson chi- 
square test of independence, but are used in cases where respondents are asked to “pick all” 
responses that apply rather than “pick any”; as such they test whether the response distributions 
of a question differ among different groups of respondents (Bilder, Loughin, & Nettleton, 2000). 
Tests of simultaneous pairwise marginal independence (SPMI) were conducted to probe the 
associations between various MRCVs. Both MMI and SPMI use a modified Pearson chi-square 
statistic to adjust for within-subject dependence among responses using non-parametric bootstrap 
procedures (Bilder & Loughin, 2004; Bilder, Loughin, & Nettleton, 2000). Post hoc analyses to 
the MMI and SPMI tests consisted of conducting Pearson’s chi-square tests and subsequent 
examinations of the contingency tables. All analyses were conducted using R v.3.0.2. (R Core 
Team, 2013). MMI and SPMI tests were performed using package “MRCV” (Kozoil & Bilder, 
2013). Chi-square tests were computed using package “gmodels” (Wames, 2013) as part of the 
statistical computing program R (R Core Team, 2013). All analyses were tested for significance 
at the p < 0.05 level. 


Results 

Of the 463 survey invitations 100 responses were received. Of these responses, 56 
surveys were fully completed, which resulted in a response rate of 12%. 

After a review of the survey data generated by the respondents, a number of important 
outcomes were identified. Initially to sort the participants into levels of involvement with EL, 
participants were given four categories to choose from (Figure 2). They were asked if their 
course or courses were: entirely experiential, primarily experiential, mostly traditional, or 
entirely traditional. Participants were asked to base their assessment on the nature of a single 
course, or if they taught more than one EL course to give answers best suited to all of their 
courses in general. The last category (entirely traditional) was added in case any individuals were 
misidentified as being engaged in experiential education. It is clear based on the results that most 
of those who participated the survey were engaged in EL to a high degree with 75% of 
respondents placing themselves in the first two categories. 
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Figure 2. Percentage of respondents (n = 56) in each course type indicating the level of involvement with EL 
methods used by survey participants. 

The above classification was then used to examine each of the participant’s work based 
on their self-assessed intensity of EL. 

Entirely Experiential 

For those who identified their courses as entirely experiential (n = 6), a majority (83%) 
described their course(s) as experiential education where students learn from direct experience. 
The next highest descriptor was active learning (67%) were students engaged in learning 
material using a diverse set of methodologies and strategies. Field-based learning (50%) rounded 
out the top three where students were working away from the institution and were in direct 
contact with the content they were studying. 

Within the entirely experiential group (see Figure 3), presentation was identified as the 
most common form of assessment (100%). Paper/essay was second (67%) and journal 
assessment (50%) and debriefing interview (50%) were the third most used assessment forms. A 
majority of the assessment methods were conducted on individuals as opposed to groups of 
students. Only presentation had a higher rating when it came to group work. No instructors 
reported using formal written examinations. 


http://ir.lib.uwo.ca/cj sotl_rcacea/vol6/iss3/4 


10 



Yates et al.: Surveying Assessment in Experiential Learning 



Assessment Method 


Figure 3. Percent of survey participants teaching entirely experiential courses (n = 6) utilizing various assessment 
methods in EL courses. 


When asked how they evaluated the assessments, a majority of respondents (67%) said 
that expert opinion was what they used (see Figure 4). 
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Grading Method 

Figure 4. Percent utilization of grading method across course types. 
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Primarily Experiential 


For those who identified their courses as primarily experiential (n = 36), respondents 
reported a combination of 13 different instructional approaches. The majority of respondents 
(94%) reported that they used direct experience as an experiential component (experiential 
education). Others that were mentioned often included field projects (75%), active learning 
(72%), presentations (67%), problem-based learning (61%), and group projects (61%). 
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The reported assessment methods for the above mentioned strategies were again very 
diverse with 18 different forms of assessment being utilized (see Figure 5). The top three 
assessment methods were presentation (67%), project report (61%), and paper/essay (50%). 
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Assessment Method 


Figure 5. Percent of survey participants teaching primarily experiential courses (n = 36) utilizing various assessment 
methods in EL courses or course components. 

As was the case with entirely experiential courses, most of the assessment methods were 
completed individually. Only presentations and role-playing had higher levels of group 
assessment when compared with individual assessment. Rubrics (78%), expert opinion (64%), 
and checklists (42%) were the most common means of assessing the experiential evaluations (see 
Figure 4). 

Partially Experiential 

For those who identified their courses as partially experiential (n = 14), respondents 
reported to use a wide range of instructional approaches. The most common instructional 
approach was experiential education (100%), followed by active learning (64%) and field 
projects (43%) as well as group projects (43%). Many assessment methods were used 
encompassing a variety of categories (see Figure 6). Paper/essay was the most commonly used 
instructional approach (79%), followed by written exams (71%), project reports (43%), 
presentations (43%), and other methods (43%). 
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Figure 6. Percent of survey participants teaching partially experiential courses (n = 14) utilizing various assessment 
methods in EL courses or course components. 

A majority of the assessments were carried out individually. The most common 
evaluation of the EL methods by this group of instructors was by rubric (64%), answer keys 
(50%), checklist (43%), and expert opinion (43%) (see Figure 3). 

Choice of Assessment Methods 

When respondents reported that they test experiential and traditional course components 
separately, written examination (65%) and paper/essay (48%) were most frequently used to 
assess non-experimental course component(s) (n = 23). Other traditional assessment methods 
such as multiple choice examinations, and experiential methods such as project reports and 
presentations, were also on the list. When it came to grading these evaluations, answer keys 
(52%), rubrics (48%), and expert opinion (35%) were the most popular. In contrast, when 
questioned exclusively about methods used to assess the experiential course component(s) (n = 
29), presentations (52%), paper/essay (45%), and project reports (41%) were most commonly 
used. Grading was most often performed using rubrics (66%), expert opinion (48%), checklists 
(38%), and participation (34%). 

Based on a pre-determined list, survey participants indicated a variety of assessment 
methods used in courses that are entirely, primarily, or partially experiential (see Figure 3). Of 
the experiential assessment methods included in that list, journal assessment, case-based exam, 
presentation, and project report were most common. In terms of traditional assessment methods, 
paper/essay was the most commonly used followed by written exam. Multiple choice exam was 
the least commonly used traditional assessment method and simulation/role-play was the least 
common experiential method. The number of methods indicated in the “Other” category shows 
that the range of methods is broader yet than explored by this study. Participants were asked to 
rank their choice of assessment method based on its contribution to the overall assessment of the 
student. The structure of the rankings did not allow for cross comparisons, so the use of those 
results was limited. However, all three experiential groups still identified through ranking that 
the paper/essay was part of the determination of the student grade. 
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When asked what factors influenced their choice of assessment methods, participants 
were free to choose as many factors as they deemed relevant from a list provided. The response 
indicated that they were most often guided by what was the common practice in their area of 
teaching (see Figure 7). Consistency, time, and objectivity in assessment were also important 
factors. 
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Figure 7: Percent of all survey participants in each course type indicating the factors influencing their choice of 
assessment method in EL courses or course components. 


A comparison of assessment methods reported across course types (see Figure 8) shows a 
trend in the decline in use of presentation, debriefing interview, and journal assessment as the 
level of EL declines across the course types. The opposite trend is seen in the use of written 
exam and multiple choice exam, which were absent in the entirely experiential course type. 
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Figure 8. Comparison of assessment methods across course type. 

Interactions between Assessment Methods Utilized 

Results of MMI tests suggest that the choice of assessment method utilized did not differ 
based on course type (k^= 41.87,p = 0.1236). However, choice of assessment methods did differ 
between experiential and non-experiential course components = 59.68, p < 0.001). When 
assessing non-experiential course components, educators were more likely to utilize written 
examinations (X? = 10.85, p <0.001) and less likely to utilize case-based exams (X} = 5.85, p = 
0.0155), practical exams/performance tests (A,^= 4.6, p = 0.0312), and presentations (X^= 4.06, p 
= 0.0179) than when assessing experiential course components. Grading methods also differed 
between traditional and experiential course components (A,^ = 14.92, p = 0.0295), with answer 
keys used more frequently in traditional course components (A,^ = 6.13, p = 0.0132). Neither 
course type (A,^ = 4.59, p = 0.3192) nor course component (A,^ = 3.32, p = 0.109) influenced 
whether educators chose to use group or individual exams. Likewise, the use of self-, peer-, and 
collaborative assessment was independent of both course type (A,^= 9.12,p = 0.2961) and course 
component (A,^= 4.02,p = 0.3687). 

Results of SPMI tests indicate that instructional approach (e.g., field-based, internship) 
did not influence the assessment methods utilized in fully experiential courses (A,^ = 29.04, p = 
0.5238), while educators implementing different instructional approaches in partially and 
primarily experiential courses did utilize different assessment methods (X? = 366.92,p = 0.003). 
Written exams were less likely to be used in a number of EL settings, including field projects (A,^ 
= 11.26,p < 0.001), problem-based learning {X}= 6.63,p = 0.0100), active learning (A,^= 8.57,p 
= 0.0034), experiential education (A,^ = 6.52, p = 0.0107), presentations (A,^ = 7.68, p = 0.0056), 
and group projects (A,^ = 5.24, p = 0.0221). In addition, active learning (A,^ = 4.88, p = 0.0271), 
experiential education (A,^ = 5.33, p = 0.0209), and presentation (A,^ = 11.09, p < 0.001) 
instructional approaches were more likely to incorporate presentations as a form of assessment. 
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Instructional approaches described as experiential education were less likely to utilize practical 
exams/performance tests (k^= 3.95,p = 0.0469). 

Simultaneous pairwise marginal independence tests were also utilized to determine if 
differences existed between the various assessment methods and grading strategies. The use of 
various grading tools (i.e., rubric, checklist) was not significantly related to the choice of 
assessment method (k^ = 102.92, p = 0.4797). However, the decision to utilize group or 
individual assessment was independent of the assessment method utilized (A,^ = 60.07, p = 
0.0157). Presentations (A,^ = 4.05, p = 0.0441) and narrative assessments (A,^ = 4.81, p = 0.0283) 
were less likely to be used in conjunction with individual exams, while written exams (A,^= 7.13, 
p = 0.0076) and “other” assessments (A,^ = 7.13, p = 0.0076) were more likely to utilize group 
exams. Assessment methods also differed with respect to their propensity to incorporate self-, 
peer-, and collaborative-evaluation (A,^ = 150.3, p < 0.001), with presentations and narrative 
assessments more likely to incorporate one of these three evaluation methods; project reports 
were less likely to use self-assessment (A,^= 7.10, p = 0.0072). In contrast, grading tools were not 
influenced by assessment type (A,^ = 102.92, p = 0.4797). Interestingly, factors that educators 
indicated as influencing their choice of assessment method(s) (including time, resources, student 
numbers, student expectations, common practice, consistency, objectivity, and other) were found 
to be independent of the assessment method utilized (A,^ = 143.07, p = 0.3727). 

Discussion 

Based on the data analysis a number of interesting outcomes related to assessment in 
experiential courses emerged. Participants could clearly identify with one of the three categories 
of EL instruction: (a) those whose courses were solely experiential, (b) those whose courses were 
primarily experiential, and (c) those whose courses were partially experiential. These findings 
supported the confidence in the participant population that they were able to respond to questions 
in an informed manner. The lack of response in the fourth category, entirely traditional, was 
important because it allowed us to confirm that all participants were indeed practitioners of EL 
as claimed and could speak to the subject based on actual experience. 

The responses in each of the three categories of experiential course type shared some 
similar characteristics. The results within each category would indicate overall that participants 
used multiple assessment methods within a single course, with instructors utilizing an average of 
five different methods. 

Eor those who reported that their teaching was entirely experiential, 74% of respondents 
used multiple EL teaching methodologies in a single course. Although the number of entirely 
experiential courses was low (n = 6), when combined with primarily experiential courses, a 
significant number have a strong experiential component (n = 38). This is evidence to support the 
idea that rather than simply dabbling, those who use EL are reporting a deep level of 
commitment to the methodology and connected to that is a recognition that the assessment 
methods used must be more matched to the method of learning. 

Eor those who worked with courses that were primarily experiential, three definite 
assessment trends emerged. Most instructors defined their instruction under broad categories 
such as experiential education, active learning, and internship/practicum; but when it came to the 
tools they used, this group had the widest range of assessment methods. They supported their 
assessment with rubrics (77.8%), expert opinion (63.9%), checklist and answer key (41.7% 
each).The entirely experiential group has eliminated most of the traditional assessment tools. 
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Only the research paper/essay was still being used. The more commonly reported assessment 
methods allowed for more student engagement and interaction with the other learners and 
students in the form of oral examination, presentations, interviews, and journaling. Also of note 
is that oral examination was used most frequently in course considered entirely experiential 
couple with a lower usage of project reports compared to other categories. This finding supports 
the notion that, in addition to innovation in the course delivery, these faculty are also innovative 
in their assessment methods. 

When primarily or partially experiential courses were assessed, instructors utilized 
different assessment methods for the traditional and experiential course components. When 
assessing traditional course components, instructors tended to use more traditional assessment 
methods, such as written examinations, and were less likely to utilize assessments that were more 
experiential in nature, such as case-based exams, practical exams/performance tests, and 
presentations. Experiential course components were more likely to utilize these experiential 
assessment methods, indicating that there is an alignment between teaching methodology and 
assessment methods. Due to the nature of the more traditional exams used in traditional course 
components, these assessments were more likely to utilize answer keys in the grading process. 
Interestingly, neither the use of self-, peer-, or collaborative-assessment, nor the use of group or 
individual exams, differed between experiential and traditional course components, perhaps 
suggesting that these assessment strategies have been effectively incorporated into more 
traditional course components. 

No differences were found between the levels of experiential education provided in a 
course (entirely, primarily, or partially experiential) and the various factors assessed in the 
survey, including assessment methods, grading strategies, the incorporation of self-, peer-, and 
collaborative-assessment, and the use of individual and group exams. This suggests that 
experiential education, regardless of the degree of its incorporation into a course, is qualitatively 
similar with respect to assessment. 

Differing instructional approaches led to the selection of different assessment methods in 
primarily and partially experiential courses. More generally defined instructional approaches 
(e.g. problem-based learning, active learning, and experiential education) as well as field 
projects, presentations, and group projects were less likely to utilize written exams. Active 
learning, experiential education, and presentation-based courses were more likely to use 
presentations as assessment methods. Such differences indicate that assessment methods are not 
applied as a blanket solution to experiential courses, and that experiential courses are 
qualitatively different in terms of what assessment methods are utilized. 

While the various assessment methods under study did not differ in terms of the tools 
used in grading (i.e., rubric, checklist), they did differ in their incorporation of group versus 
individual exams as well as the use of self-, peer-, and collaborative assessment. Presentations 
and narrative assessments were less likely to be used as an individual assessment, and were more 
likely to incorporate some form of student assessment, indicating that these assessment methods 
are often viewed by instructors as a cooperative form of evaluation. 

Conclusion 

Given that the sample size was small, we do not profess to suggest an ideal assessment 
method for EL. What we do see is the potential for replication of the study on a larger scale or at 
other institutions to continue to inform the development of EL. The survey does indicate that the 
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most commonly used forms of assessment in EL courses are journal, case-based exam, project 
report, and presentation. The most common traditional assessment tool used was paper/essay. 
Although the results indicate that courses with a primary or partial EL component are less likely 
to use written examination as an assessment tool and more likely to use experiential based 
assessment tools such as presentations, what is unknown is what impact the compatibility or lack 
thereof between teaching method and assessment tool has on student performance and the 
student experience. Some such matches are logical, such as assessing understanding obtained 
through participating in a case study with a case-based exam or a practical exam to test the 
learning of applied field skills. However, is assessment via presentation appropriate for problem- 
based learning or other forms of experiential education? Understanding the relationship between 
assessment type and the learning experience would be valuable for scaling up EL assessment to 
an institutional level. 

As the main purpose of this research was to create an informed overview of the EL 
landscape at our university, we feel we have taken a successful first step by defining and 
quantifying the use of various EL assessment methods. Our research team felt strongly that 
surveying faculty was the most effective method of creating an inventory of what is taking place 
currently. Based on what we have learned we continue to believe this to be true. The data and our 
findings have the potential to be turned around quickly and shared with those currently engaged 
in EL and those wishing to move into a more engaged form of teaching and learning. The 
process also allows us to dig deeper into the world of EL knowing that we have connect with a 
knowledgeable set of practitioners. 

Also, it is important to learn how connected or disconnected faculty who deliver EL are 
to a variety of traditional and non-traditional forms of assessment. Some traditional assessment 
methods are nowhere to be found where some, such as the venerable essay, still have a place in 
the assessment toolbox of those delivering EL. Institutions need to make a concerted effort to 
promote quality assessment in all teaching and learning situations. Eaculty with little or no 
understanding of EL assessment who are venturing into EL teaching methodologies may not 
achieve success and their students may not achieve learning outcomes. 

The advice we provide to instructors revolves around preparation for change. The biggest 
adjustment instructors may face is letting go of traditional assessments such as multiple-choice 
examinations. Time is required to prepare and variety in the evaluation and the instructional 
processes is common. It will be critical to match the instructional process with the proper or 
logical assessment tool regardless of the level or mix of EL teaching being used. Incorporating a 
wide variety of methods of assessment will assist in gathering a range of data and judging the 
efficacy of instructional methods one uses. As we will share our findings with those who are 
interested in improving assessment, we do not offer prescriptive advice into teaching strategies 
outside of identifying what is reportedly taking place, but we do believe there is a strong link 
between changing both process and evaluation at the same time. 

Institutional tracking of all forms of assessment allows for the identification of areas 
where EL assessment methods may enhance instruction. This initial step is important in 
advancing major efforts to expand EL teaching and learning. Our research findings support the 
need to study the connection of teaching methods with compatible assessment methods. In future 
research, students might be canvassed to determine if they share similar attitudes towards EL 
assessment methodologies and how effective the main tools for assessment are. In this way, the 
relationship between learning experience and assessment tool in terms of the best match can be 
explored for its impact on learning outcomes and the student experience. We plan to use the 
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results of this research to guide the development of a pilot study for the purpose of testing 
differences between the most commonly used methods of assessment in terms of student learning 
and experience. 
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Appendix 

The Survey 


1. Would you describe this course as: 

a) Entirely experiential 

b) Primarily experiential with some traditional components 

c) Primarily traditional with some experiential components 

d) Entirely traditional 

If respondents answered “Entirely experiential” to question 1, the following questions were then 
asked: 

2. Using up to ten keywords, describe the type(s) of experiential learning employed in your 

course. 

3. Which of the choices below best describes the type of experiential learning provided in 

your course: 

a) Active learning - students engage the material they study through reading, writing, 
talking, listening, and reflecting; examples include debates, role-playing, 
brainstorming, problem-solving, etc. 

b) Case studies - students take on the role of decision makers through written 
descriptions of problems or situations 

c) Cooperative education - students simultaneously attend classes part-time and work 
part-time during consecutive school terms in an intentionally planned and coordinated 
way 

d) Experiential education - students learn from direct experience 

e) Eield-based - supervised student research or practice carried out away from the 
institution and in direct contact with the people, natural phenomena, or other entities 
being studied 

f) Independent study - students pursue an area of interest in their field with faculty 
guidance 

g) Internship/practicum - students engage in practical experience in a work setting as 
well as theoretical study 

h) Problem-based learning - student work cooperatively in groups to seek solutions to 
real world problems 

i) Service learning - students participate in an organized service activity that meets 
some community need and student reflection increases understanding of course 
content 

j) Other, please specify... 

4. Using up to ten keywords, describe the type(s) of assessment methods utilized in your 

course. 
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5. From the following list, select the method(s) of assessment utilized in your course. Check 
all that apply: 

a) case-based examination 

b) concept mapping 

c) debriefing interview 

d) journal assessment 

e) multiple-choice examination 

f) narrative assessment 

g) open-book examination 

h) oral exam 

i) paper/essay 

j) portfolio assessment 

k) practical examination/performance test 

l) presentation 

m) product assessment 

n) project report 

o) simulation/role-playing 

p) take-home examination 

q) written examination 

r) other, please specify... 

For each method selected in question 5, the following are asked 

6. Please indicate whether the_is performed as an individual exercise or in a 

group: 

a) Individual 

b) Group 

7. Which of the following do you use to determine a grade when using the_ 

method? 

c) Answer key 

d) Checklist 

e) Expert opinion 

f) Participation 

g) Rubric 

h) Other, please specify... 

The survey they continues as follows: 


' Note that due to the low number of respondents for these questions partitioned by assessment method, all responses 
were bulked for the MMI and SPMI statistical analysis. 
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8. Of the assessment methods you have identified, rank them in order of their contribution 
to the overall student grade, with one representing the highest contribution. 

9. Do you provide students with a description of your expectations prior to assessment (ex: 
distribution of rubrics, advanced organizers, planning guides, or links to example 
questions)? 

a) Yes 

1. Briefly describe how you provide students with a description of your expectations. 

b) No 

10. Which of the following assessment strategies do you utilize? Check all that apply: 

a) Self assessment 

b) Peer assessment 

c) Collaborative assessment 

d) None of the above 

11. What factors influence your choice of assessment method(s)? Select all that apply: 

a) Common practice in your area of teaching 

b) Consistency 

c) Objectivity 

d) Resources 

e) Student expectations 

f) Student numbers 

g) Time 

h) Other, please specify... 

End. 

If respondents answered “Primarily experiential with some traditional components” or “Primarily 
traditional with some experiential components” to question 1, question 3 (above) would be 
modified to instead list the following choices: 

a. Active learning - students engage the material they study through reading, writing, 
talking, listening, and reflecting (ex: debates, role-playing, brainstorming, etc.) 

b. Experiential education - students learn from direct experience 

c. Eield project - an experience in which students are in direct contact with the people, 
natural phenomena, or other entities being studied; students complete a project based on 
their experience in the field setting 

d. Eield trip - an experience in which students are in direct contact with the people, natural 
phenomena, or other entities being studied 

e. Group project 

f. Independent study - a student-initiated experience planned to allow students the 
opportunity to pursue an area of interest in their field with faculty guidance 
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g. Laboratory 

h. Oral interviews 

i. Presentation 

j. Problem-based learning - instructional method that challenges students to "learn to learn," 
working cooperatively in groups to seek solutions to real world problems 

k. Tutorial 

l. Use of primary source or raw data 

m. Other, please specify... 

Participants were then asked “Do you test the experiential learning component(s) of your course 
separately from the non-experiential learning component(s)?” If respondents answered “No” to 
this question, they would continue with questions 4 through 11 as outlined above. If respondents 
answered “Yes” to this question, they would be asked questions 4 through 11 as outlined above 
for the experiential course component and the traditional course component separately^. 

If respondents answered “Entirely traditional” to question 1, the survey would end immediately. 
All willing survey participants were asked to share assessment materials, such as course syllabi. 


^ For calculation of MMI and SPMI statistics, responses for course components were bulked together for all analysis 
with the exception of those that explicitly examined differences between traditional and experiential course 
components. 
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